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Abstract 

This study sought to validate a new measure, the Classroom Cohesion Survey (CCS), designed 
to examine the relationship between teachers and classroom assistants in autism support 
classrooms. Teachers, classroom assistants, and external observers showed good inter-rater 
agreement on the CCS and good internal consistency for all scales. Simple factor structures were 
found for both teacher- and classroom assistant-rated scales, with one-factor solutions for both 
scales. Paired t tests revealed that on average, classroom assistants rated classroom cohesion 
stronger than teachers. The CCS may be an effective tool for measuring cohesion between 
classroom staff and may have an important impact on various clinical and implementation 
outcomes in school settings. 
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Since the Individuals with Disabilities Education Act was passed in 1990, the number of children 
with autism spectrum disorders (ASD) served in the public school system has increased by 13% 
to 28% each year and by more than 1700% from 1992 to 2008 (Maenner & Durkin, 2010). 
Schools are under increasing pressure to effectively incorporate evidence-based interventions for 
children with ASD into special education classrooms (National Research Council, 2001). These 
interventions are complex and involve using multiple instructional strategies concurrently and 
generally have not been effectively implemented in community settings (Dingfelder & Mandell, 
2011; Stahmer, Codings, & Palinkas, 2005). 

Implementing these complex intervention strategies with fidelity in under-resourced urban 
schools requires a collaborative effort among the multiple staff members who provide behavioral 
and educational support in special education classrooms (Dingfelder & Mandell, 2011). More 
than 75% of special educators report that they supervise assistants and one-to-one aides (French, 
2001). Studies find that classroom assistants perform their duties most effectively when (a) they 
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are appropriately supervised, (b) their roles are clearly defined, (c) they are trained for assigned 
tasks, and (d) they participate in regularly scheduled planning meetings (French, 2001). Teachers 
typically receive little or no training in how to interact with their aides, and little is known about 
the quality and impact of staff relationships in these classrooms (French, 2001; Scheuermann, 
Webber, Boutot, & Goodwin, 2003). 

In other fields, conceptual models from organizational psychology have been used to improve 
our understanding of team functioning (Wang, Ying, Jiang, & Klein, 2006). Much like in other 
settings, teachers and classroom assistants must function as a team, defined as “a collection of 
individuals who are interdependent in their tasks” (Cohen & Bailey, 1997, p. 241). Team func¬ 
tioning, which can include a number of dimensions such as collaboration, conflict resolution, 
leadership, and cohesion, has been shown to affect both performance and outcomes across a 
broad range of settings (Lemieux-Charles & McGuire, 2006). Cohesion, the tendency of a group 
to “stick together and remain united in the pursuit of its goals and objectives” (Wang et al., 2006), 
is most closely associated with team performance and outcomes (Lemieux-Charles & McGuire, 
2006). 

Although studies have examined the impact of staff cohesion in general on job performance 
and intervention implementation, no published study has addressed cohesion among staff in spe¬ 
cial education classrooms, where many children with autism spend the majority of their time 
(National Research Council, 2001). This dearth of research focused on special education settings 
may be due in part to the fact that no measures have been specifically developed for special edu¬ 
cation settings. The few studies of cohesion in academic settings—and measures developed for 
these settings—have focused on cohesion among students (e.g., Rovai, 2002). The majority of 
other cohesion measures have been developed for use in business or health care settings. Although 
these existing cohesion questionnaires, such as the Perceived Cohesion Scale (Bollen & Hoyle, 
1990), may capture general aspects of the classroom dynamic, such as a sense of belonging and 
morale, they do not capture task-specific performance issues that are relevant to the special edu¬ 
cation classroom, such as the teachers’ role in providing effective supervision of paraprofession- 
als. This is important because performance is predicted most strongly when there is a match 
between the dimensions of cohesion and performance (Chang & Bordia, 2001). Existing mea¬ 
sures that tap into more task-specific performance tend to focus on individuals at the same level 
within a group (e.g., Group Environment Questionnaire; Carron, Widmeyer, & Brawley, 1985; 
Chang & Bordia, 2001), rather than a supervisor-supervisee relationship with different levels of 
training and different roles. Developing tools that capture this hierarchical relationship is an 
important first step in advancing our study of how the special education classroom staff cohesion 
affects intervention implementation. 

We recently completed a randomized-controlled field trial of an evidence-based behavioral 
intervention for children with ASD, which required classroom staff to deliver a proven-effica¬ 
cious comprehensive intervention package. Strategies for Teaching based on Autism Research 
(STAR; Arick, Loos, Falco, & Krug, 2004), to children in autism support classrooms in a large, 
urban school district (Mandell et al., 2013). During the study, we became aware of significant 
variability in the degree to which teacher and classroom assistants considered themselves part of 
a team working toward a common goal. We observed that some classrooms that functioned as a 
cohesive unit were better able to implement STAR, whereas other classrooms with more frag¬ 
mented teams had more difficulty. This observation that the relationship among classroom staff 
may be associated with the implementation of evidence-based practices in classrooms led us to 
develop a tool that measures classroom staff cohesion. We hypothesize that classroom cohesion 
will be associated with two factors: (a) teacher burnout or stress, with teachers who experience 
higher levels of burnout having poorer working relationships with their classroom assistants and 
lower classroom cohesion, and (b) STAR program fidelity, in which higher classroom cohesion 
will be associated with higher program fidelity. In this article, we present this measure and 
explore its psychometric properties. 
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Table 1 . Teacher and Classroom Assistant Baseline Characteristics: Descriptive 

Statistics. 


Teachers 

Classroom assistants 


(n = 149) 

(n = 154) 

Female (%) 

79.9 

81.1 

Race/ethnicity (%) 



African American 

18.2 

57.9 

Latino 

1.3 

6.9 

Caucasian 

69.8 

13.2 

Asian 

1.3 

1.3 

Multiethnic 

0.6 

2.5 

Other 

0 

2.5 

Unknown 

8.8 

15.7 

Age in years (M, SD) 

38.96 (1 1.97) 

45.23 (1 1.16) 

Years experience in classroom (M, SD) 

1 1.24 (10.40) 

4.18 (5.59) 

Years experience in special education (M, SD) 

9.90 (10.10) 

7.64 (7.91) 

Years experience with autism (M, SD) 

4.41 (5.99) 

5.66 (5.64) 

Bachelor’s degree or higher (%) 

100 

22 


Method 

Participants 

Data were drawn from the 3-year field trial described above (Mandell et al., 2013) and from a 
2-year District and University fee-for-service contract for the 2011-2013 school years. The cur¬ 
rent study includes data from teachers (n =149) and classroom assistants (« = 154) who partici¬ 
pated in either Year 3 of the study or Year 4 or 5 as a fee-for-service contract. Table 1 provides a 
description of the sample. A sub-sample of teacher ( n = 53) and classroom assistant (n = 53) data 
from Year 3 were used for additional analyses. 

Measures 

Classroom Cohesion Survey (CCS; teacher and classroom assistant versions). The CCS was developed 
and revised in multiple stages. In the first stage, the literature on cohesion was reviewed. From 
this literature review, a preliminary survey addressing gaps in the cohesion literature pertinent to 
our classroom observations from the randomized trial was constructed. A total of 20 items were 
initially generated for review. In the second stage, the preliminary survey was distributed to 
teachers from a nonparticipating district to ensure its straightforwardness and relevance. In the 
final stage, two items that were confusing or deemed irrelevant were revised or eliminated. These 
included questions related to additional support staff (e.g., one-to-one instructional assistants 
from outside behavioral agencies) and negative feelings toward classroom staff. The CCS- 
Teacher version is an 18-item self-report of the working relationship between the teacher and 
classroom assistant. The CCS-Classroom Assistant version is also an 18-item self-report with 
items that parallel the teacher version. Responses from both versions are structured on a 5-point 
Likert-type scale (1 = not true at all, 2 = hardly ever true, 3 = true sometimes, 4 = true most of 
the time, 5 = always true). 

Maslach Burnout Inventory (MBI)-Education Form (Maslach, Jackson, & Leiter, 1996). To examine 
convergent validity with cohesion, teacher burnout was measured using the MBI, a 22-item 



332 


Journal of Psychoeducational Assessment 33(4) 


self-report inventory designed specifically for diagnosing burnout and job stress in teachers. It is 
composed of three subscales: Emotional Exhaustion (EE), Depersonalization (DP), and Personal 
Accomplishment (PA). Responses are structured on a 7-point Likert-type scale ranging from 0 
(feeling has never been experienced) to 6 (feeling is experienced daily). Scores were derived 
using item-level means for each subscale (ranging from 0 to 6). The MBI was available for the 
53 teachers in Year 3 of the randomized controlled trial. 

Program fidelity. For teachers included in Year 3 of the study (n = 53), video observations were 
coded to assess the teachers’ fidelity of implementation of each component of STAR. The STAR 
program combines three instructional approaches based on the principles of applied behavior 
analysis: discrete trial training (DTT), pivotal response training (PRT), and functional routines 
into a comprehensive curriculum for children with ASD. DTT is a highly structured, one-on-one 
teaching strategy that breaks complex behaviors into smaller teachable units (Arick et al., 2004), 
whereas PRT uses a more naturalistic, play-based approach to target crucial skills, including 
motivation and responsivity to the environment that are pivotal for many other skills (Koegel et 
ah, 1989). Functional routines are predictable events that involve a chain of behaviors and are 
implemented in the context of daily classroom activities. Each teacher was filmed for 30 min (10 
min of each of the three core intervention strategies) once per month. We selected the latest fidel¬ 
ity observation period (occurring between March and May of 2011) to examine the predictive 
validity of the CCS. Research assistants, who were blind to the research hypotheses and teachers’ 
training group or experience level, were trained to code the video samples using a set of behav¬ 
ioral definitions for each component of STAR. Measures for each component of the STAR pro¬ 
gram were designed based on the manuals and in consultation with the program developers to 
ensure teachers implemented the program in the way that it was designed (Mandell et al., 2013). 
Coders rated the use of each component of STAR on a 1 to 5 Likert-type scale after viewing the 
entire video clip. The coding criteria differed for each STAR intervention strategy. A low score 
of 1 indicated the teacher did not use the strategy during the session or never implemented it cor¬ 
rectly. A high score of 5 indicated the teacher implemented the component competently through¬ 
out the segment. To create a global rating of fidelity, the percentage of trials in a single videotaped 
session that earned a passing score (in which 75% of the steps were implemented competently 
and coded with a 4 or 5) was calculated. All scores for each program component were averaged 
across all months of the academic year to create a cumulative measure of STAR program fidelity 
throughout the academic year. 

Procedure 

Teachers and classroom staff were recruited through the District’s Office of Specialized Services. 
The District sent each staff member a letter describing the project and asking him or her to par¬ 
ticipate. In Year 3, the classroom consultants were graduate and postdoctoral-level fellows in 
school and educational psychology with training in applied behavior analysis recruited from a 
local university. Some teachers and classroom staff were new to the STAR program in Year 3, 
whereas others had received previous training in Years 1 and 2, but all received the same training 
in Year 3. This included didactic instruction (intense professional development workshops 
throughout the school year) and in vivo coaching (45-60 min sessions that included direct obser¬ 
vation, modeling, and performance feedback) that occurred every 2 weeks. Teachers were asked 
to facilitate DTT and PRT sessions as well as functional routines throughout the day with their 
students to enhance academic, language, and social outcomes. Teachers were asked to complete 
a battery of assessments that included demographic measures, the CCS, and MBI. As an incen¬ 
tive, teachers received a small stipend for completing these measures. Classroom assistants 
attended professional development workshops with the teachers, and all classroom assistants 
were invited to participate during in vivo coaching. However, classroom assistants varied in their 
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participation during in vivo coaching. Some classroom assistants were highly involved in deliv¬ 
ering instruction to students and were asked to facilitate DTT and PRT sessions and functional 
routines. Other classroom assistants did not actively participate in consultation and were not 
involved in delivering instruction to students, but instead took the role of monitoring students 
while the teacher provided instruction. Classroom assistants also were asked to complete demo¬ 
graphic measures and the CCS. 

During Years 4 and 5 as a fee-for-service contract, classroom consultants were full-time staff 
members with a minimum of a master’s degree in school psychology, applied behavior analysis, 
or education. Training in Year 4 included didactic instruction and in vivo coaching. Data were 
included for classrooms that were new to STAR. Didactic instruction (professional development 
workshops) and in vivo coaching (60-90 min sessions that included direct observation and mod¬ 
eling) occurred twice per month. In Year 5, all teachers receiving consultation were new to the 
STAR program. Didactic instruction occurred once for teachers and twice for classroom assis¬ 
tants, and in vivo coaching (60-90 min sessions that included direct observation and modeling) 
occurred once every 2 to 3 weeks. 

Statistical Analyses 

All analyses were conducted using PASW software version 20.0. First, exploratory principal axis 
factor analyses were conducted to establish the best factor solution for the teacher- and classroom 
assistant-rated CCS. We sought factor solutions that would meet the following criteria: (a) satisfy 
Cattell’s (1966) scree test, (b) retain at least three items per factor with salient loadings (> .40), (c) 
yield high internal consistency for the scale with salient items included (> .70), (d) achieve a 
simple structure with a maximum number of items retained, (e) produce a Kaiser-Meyer-Olkin 
(KMO) measure of sampling adequacy greater than 0.5, and (f) yield a statistically significant 
(p < .001) Bartlett’s test of sphericity. We then calculated a total score for the CCS by averaging 
the items of the measure for each type of respondent (i.e., teachers and classrooms assistants) and 
each factor. Next, we examined whether there were systematic within-classroom differences 
between teachers and classroom assistants in these scores by comparing the teacher and assistant 
scores using paired-sample t tests. We then examined inter-rater agreement by calculating Pearson 
correlations among the raters (i.e., teachers and classroom assistants). Finally, we examined the 
convergent validity for the CCS by examining its correlations with program fidelity and burnout. 

Results 

Factor Analysis 

Separate exploratory principal axis factor analyses were conducted to determine the latent struc¬ 
ture of the teacher- and classroom assistant-rated CCS. The KMO measure of sampling adequacy 
(0.96 and 0.94, respectively) and Bartlett’s tests of sphericity (2778.76 and 2467.28, p < .001, 
respectively) indicated that the tests were significant for both the teacher- and classroom assistant 
rated CCS. The teacher- and classroom assistant-rated CCS yielded unitary factors. Overall, 68% 
and approximately 60% of the total variance were explained on the teacher- and classroom assis¬ 
tant-rated CCS, respectively. The factor loadings of each item on the teacher and classroom 
assistant version of the CCS are presented in Tables 2 and 3. 

Descriptive Data 

Teachers had an item-level mean of 4.16 (SD = 0.82) on the CCS, whereas classroom assistants 
had an item-level mean of 4.41 (SD = 0.72). Teachers from Year 3 generally reported low-to- 
average levels of burnout on the MBI, with item-level means of 2.09 (SD = 1.33) on emotional 
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Table 2. Item-Level Factor Analysis of Cohesion—Teacher Form (n = 149). 


Items Factor I 


1. In general, I can rely on my classroom assistant when I need help. .86 

2. I am comfortable delegating tasks to my classroom assistant. .81 

3. My classroom assistant and I act like a team. .92 

4. I am satisfied with the amount of support I receive from my classroom assistant. .89 

5. My classroom assistant has a schedule that we agreed upon that he/she follows every .73 

day. 

6. My classroom assistant and I agree on the best ways to work with our students. .89 

7. My classroom assistant knows what he/she needs to do without my having to ask him/ .88 

her. 

8. My classroom assistant has good ideas to improve our classroom’s functioning. .82 

9. I often feel like I am working alone in instructing my students. .52 

10. lam open to the suggestions from my classroom assistant. .71 

I I. I trust my classroom assistant to do the tasks for which he/she is responsible. .85 

12. When I feel frustrated or overextended, I can rely on my classroom assistant for .88 

support. 

13. I successfully lead the staff who work in my classroom .64 

14. If I experience challenges with a student, my classroom assistant provides valuable .88 

help. 

15. The “burden” and everyday stressors of our work are shared by my classroom .78 

assistant and me. 

16. My classroom assistant and I meet regularly to discuss strategies for working with our .82 

students (e.g., what’s working, what’s not, etc.). 

17. My classroom assistant agrees with me about the work expected of him/her. .90 

18. When there’s a problem in my classroom, my classroom assistant asks for my advice. .78 

Alpha (a) .97 


exhaustion, 0.72 (SD = 0.99) on depersonalization, and 5.24 (SD = 0.79) on personal accomplish¬ 
ment (where higher scores indicate more burnout for the emotional exhaustion and depersonali¬ 
zation scales, but less burnout for the personal accomplishment scale). Teachers from Year 3 had 
high fidelity to DTT and functional routine strategies with means of 0.75 (SD = 0.30) and 0.76 
(SD = 0.23), respectively (indicating an average 80% pass rate of these program components 
across months), and moderate ratings of program fidelity for PRT with means of 0.58 (SD = 0.23; 
indicating an average 60% pass rate of PRT program components across months). Overall, the 
mean program fidelity was 0.72 (SD = 0.16; indicating an overall pass rate of 70%). 

Within-Classroom Concordance of Multiple Informants’ Reports 

Separate paired samples t tests were conducted to examine the within-classroom differences 
among teachers’ and classroom assistants’ scores of cohesion. Overall, classroom assistants 
assigned more positive ratings of cohesion as compared to teachers. There were statistically sig¬ 
nificant within-classroom differences between teachers’ and classroom assistants’ cohesion 
scores, 7(143) = -2.84, p = .01. 

Inter-Rater Agreement 

Pearson correlations were conducted to examine agreement among teachers’ and classroom 
assistants’ reports of classroom cohesion (Table 4). Overall, there was strong agreement among 
raters, r = .45, p < .001. 



Kratz et al. 


335 


Table 3. Item-Level Factor Analysis of Cohesion—Classroom Assistant Form (n = 153). 


Items Factor I 


1. In general, the teacher helps me when I need it. .84 

2. I am comfortable making suggestions to improve our classroom’s functioning to .72 

the teacher. 

3. The teacher and I act like a team. .91 

4. I am satisfied with the amount of support I receive from my teacher. .93 

5. The teacher and I have a schedule that we agreed on that we follow every day. .82 

6 . The teacher and I agree on the best ways to work with our students. .90 

7. It is clear what tasks in the classroom I am responsible for leading. .74 

8 . The teacher has good ideas to improve our classroom's functioning. .80 

9. I often feel like I am alone in working with the students in our classroom. .47 

10. I often introduce new strategies and ideas to work effectively with our students. .46 

I I. I trust the teacher to do the tasks for which he/she is responsible. .80 

12. When I feel frustrated or overextended, I can rely on the teacher for support. .74 

13. When we experience challenges in our classroom, I offer potential solutions to .40 

the teacher. 

14. If I experience challenges with a student, the teacher provides valuable help. .88 

15. The “burden” and everyday stressors of our work are shared by the teacher .72 

and me. 

16. The teacher and I meet regularly to discuss strategies for working with our .77 

students (e.g., what’s working, what’s not, etc.). 

17. The teacher and I agree about the work expected of me. .84 

18. When there is a problem in our classroom, the teacher asks for my advice. .70 

Alpha (a) .95 


Table 4. Correlations of Teacher and Classroom Assistant Classroom Cohesion Scores, Fidelity, and 
Teacher-Rated MBI Scores. 



1 

2 

3 

4 

5 

6 

7 8 

1. Teacher-rated cohesion, total score 

2. CA-rated cohesion, total score 

.45** 







3. Discrete trial training fidelity 

-.24 

-.13 






4. Pivotal response training fidelity 

-.01 

.12 

.24 





5. Functional routines fidelity 

-.04 

-.06 

.17 

.08 




6 . Total fidelity 

-.16 

-.05 

.62** 

.67** 

.62** 



7. MBI-Emotional Exhaustion 

-.06 

-.05 

-.19 

-.31* 

-.10 

-.29* 


8 . MBI-Depersonalization 

-.15 

-.16 

-.15 

-.14 

-.13 

-.18 

.58** 

9. MBI-Personal Accomplishment 

.09 

.17 

.18 

.16 

-.15 

.02 

_ 59 ** — 4 £** 


Note. MBI = Maslach Burnout Inventory; CA = classroom assistant. 
*p < .05. **p < .01. 


Convergent Validity 

We examined convergent validity by examining associations among teachers’ and classroom 
assistants’ ratings of cohesion and teachers’ ratings of burnout and fidelity of program implemen¬ 
tation for each component of STAR. Teacher- and classroom assistant-rated cohesion were not 
statistically significant with measures of fidelity or burnout (see Table 4). 
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Discussion 

This study presents preliminary evidence for the factor structure, internal consistency, and con¬ 
struct validity of the CCS scales. Analyses revealed good internal consistency for both scales (.97 
and .95 for the teacher and classroom assistant scales, respectively). We also found simple factor 
structures for both the teacher- and classroom assistant-rated scales, with one-factor solutions for 
both scales. These measures also demonstrated good variability, with a standard deviation of 0.82 
for the teacher scale and 0.72 for the classroom assistant scale (both on a 5-point scale). Our find¬ 
ings regarding convergent validity were less conclusive. There were no associations between 
cohesion ratings and ratings of burnout and fidelity, suggesting that cohesion, burnout, and fidel¬ 
ity were three distinct constructs. 

Findings regarding rater concordance suggested overall high within-classroom reliability. In 
general, teachers and classroom assistants’ reports of classroom cohesion were highly correlated; 
however, classroom assistants systematically assigned higher ratings of classroom cohesion than 
teachers. Whereas classroom assistants may have had the most positive impressions of classroom 
cohesion, an alternative explanation is that, given their role in the classroom as supervised by the 
teacher, these ratings were more vulnerable to positive respondent bias. 

Our study had some limitations that highlight the unique challenges associated with studying 
organizational factors such as cohesion in special education classrooms. First, we were limited in 
our sample size. Our sample of 149 teachers and 154 classroom assistants limits the reliability of 
our findings. For example, our factor analyses violated the general rule of thumb of having 10 
respondents per item (Kerlinger, 1986; although see Sapnas & Zeller, 2002 for a counterargu¬ 
ment). We also may have missed associations that would have been statistically significant in a 
larger sample. Finally, because there are no known measures of classroom cohesion in the special 
education literature, we had no clear measure of classroom cohesion against which to test our 
measures for convergent validity. We relied on teacher reports of burnout and fidelity of program 
implementation, but because these are different constructs, it left the findings of the lack of asso¬ 
ciations between cohesion and these other constructs somewhat difficult to interpret. We also 
only examined program fidelity via taped classroom observations, which focused primarily on 
the quality of intervention implementation (i.e., whether teachers/staff could implement each 
intervention strategy) rather than the quantity (i.e., whether teachers/staff did implement the 
intervention strategies consistently and how often they were used). It is possible that we would 
have seen a stronger, more consistent cohesion-fidelity relationship had we looked more at the 
quantity of intervention strategies delivered, which may require more consistent, coordinated 
effort. 

Despite these limitations, the results suggest the promise of a new tool to measure classroom 
cohesion. Based on our preliminary observations, this tool is relatively easy to administer with 
minimal burden to teachers and classroom assistants and fills an important need in improving our 
understanding of the working relationship among staff in special education settings. Studying 
cohesion and related areas of team functioning has led to interventions to improve team function¬ 
ing and subsequent outcomes across a broad range of settings, including business and sports 
(Weinberg & McDermott, 2010) and health care (Buljac-Samardzic, Dekker-van Doom, van 
Wijngaarden, & van Wijk, 2010) settings. Similarly, understanding classroom cohesion may 
have an important impact on various clinical and implementation outcomes in school settings and 
warrants further exploration. 

Future studies should continue to explore and refine the CCS and examine important con¬ 
structs from the organizational psychology literature (e.g., team-based measures of cohesion) 
that may be useful in providing convergent validity. Our study focused on classroom perceptions 
specific to task performance; however, cohesion is multidimensional. Broadly, cohesion typi¬ 
cally is defined in two ways: (a) as an objective attitude of the group as a whole, and (b) as a 



Kratz et al. 


337 


function of each member’s subjective perceptions of his or her own standing in the group (Bollen 
& Hoyle, 1990). Although the objective and subjective domains of cohesion are expected to be 
somewhat related, we expect that different dimensions of cohesion may predict different things. 
Examining novel ways to measure cohesion and their relative contributions in predicting both 
clinical and implementation outcomes is an important area of study. 

While considerable empirical work remains to be done to achieve a more complete under¬ 
standing of classroom cohesion and its potential utility, this study offers a unique tool to examine 
classroom cohesion in special education settings. The development of the CCS is an important 
first step in identifying and understanding variables that may be necessary and sufficient for suc¬ 
cessful implementation of classroom-wide interventions. It is our hope that the tools adapted in 
this study can be used to improve program effectiveness and sustainability in school settings, 
ultimately leading to better outcomes for children with autism. 
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